refactor(dynacell/eval): simplify the #511 prediction-provenance code - #514

The marker digest, the prediction-sources digest and the encoder-config
hash each built the same key-sorted JSON payload and truncated sha256
by hand. json_sha256_12 does it once; digests are unchanged (checked on
a real marker, a real 100-position source snapshot and a non-JSON
encoder value, which default=str still serializes).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvkM1gKifWmoDPq8Drd4JF

The bounded scandir for `.`-separated keys served only flat zarr v2
layouts, and every evaluated non-v3 prediction store uses `/`
separators, so nothing reaches it. A flat store now resolves only its
exact chunk key; when that chunk is absent the position reads as
unwritten, which rebuilds a legacy cache entry over it once. The `/`
prefix fallback and the exact-key lookup for every layout stay.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvkM1gKifWmoDPq8Drd4JF

…rces

prediction_source duplicated prediction_sources for one position; a
positions= argument covers it. The on-demand read for a position that
appeared after the snapshot now takes the store and channel from the
manifest's pred identity, which init_cache_context already seeds and
validates, so _CacheContext no longer carries its own copy of them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvkM1gKifWmoDPq8Drd4JF

The legacy manifest upgrade and the final-metrics gate each spelled out
"a position is verified iff its first stored chunk exists and is no
newer than the legacy record's time". source_predates states it once;
the upgrade checks the missing-built_at case once instead of per
position, and drops an isinstance guard on the dict it just wrote.

The upgrade also no longer builds per-family identities it discards:
_artifact_checks splits into _artifact_entries (kind, label, manifest
keys) and _artifact_identity, which only the param check needs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvkM1gKifWmoDPq8Drd4JF

…ader

metrics_provenance_matches already reads metrics_provenance.json; the
final-metrics gate then read it a second time, without the is_file()
guard, to compare the prediction digest. It now takes the current
prediction_sources and does both checks from one read, with the same
semantics: the stamped digest must match, and an unstamped sidecar is
judged by date through source_predates against its mtime.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvkM1gKifWmoDPq8Drd4JF

The five cache writers each repeated _update_manifest_entry, then
_record_position, then mark_manifest_dirty; _record_write does the three.
_record_position absorbs _add_position's separate manifest walk and uses
_manifest_leaf, and _source_current takes the entry's keys instead of
every caller walking to the entry first, which drops its isinstance
guards on values that are always dicts. _instance_keys names the
instance-label entry once for the writer, instance_cache_hit and the
family list.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvkM1gKifWmoDPq8Drd4JF

flush_manifest, the only caller, always passes the positions this
process wrote, and the source merge is only correct with them: a
defaulted empty set would let the on-disk copy win for positions this
process just rebuilt. Making the argument required keeps a future
caller from dropping it silently.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvkM1gKifWmoDPq8Drd4JF

evaluate_model and the grouped driver read every position's sources to
stamp the metrics, then init_cache_context read them all again for the
pred cache. The caller's snapshot now flows through evaluate_predictions
and init_cache_contexts into the pred context, which copies it (it adds
positions read on demand); the stamp hashes the same snapshot. The
final-metrics gate and process-executor workers still read their own.

The save_metrics / write_metrics_provenance kwarg becomes
prediction_digest: prediction_sources_sha256_12 shadowed the function
of that name. The sidecar key is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvkM1gKifWmoDPq8Drd4JF

The README still dated a position by its t=0 chunk; it is the earliest
stored timepoint since blank outputs store none at t=0. The feature
prefetch comment named force_recompute, but the guard is now any
position that is rewritten anyway. Comments that restated the
_source_current call go, and the legacy-dating docstrings keep their
reasoning without the campaign's dates and counts.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvkM1gKifWmoDPq8Drd4JF

Four places in the prediction-identity tests re-derived "zarr metadata
vs chunk file" by hand; _is_metadata and _chunks now define it once. The
pipeline cache-reuse test builds its prediction store with
make_hcs_plate like the other eval tests, dropping its own iohub setup.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvkM1gKifWmoDPq8Drd4JF

The plain libyaml SafeDumper differs from OmegaConf.save in three
ways that matter here: it leaves a str such as the hex sha12 marker
4e1234567890 unquoted, which OmegaConf's loader (and any tool that
reads a manifest with it) parses as a float; it emits &id001 aliases
when two entries share one object, as ctx.spacing does; and it
escapes non-ASCII text. A dumper subclass now ignores aliases and
single-quotes any str that parses as a bool, int or float, as
OmegaConf's own representer does, and allow_unicode is set. A new
test pins the output byte-for-byte to OmegaConf.save on those cases.

Found in the internal review of this branch; all 1224 production
manifests were already byte-identical, since none holds such a str yet.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvkM1gKifWmoDPq8Drd4JF

save_manifest truncated the manifest in place while
init_cache_context reads it without the manifest lock, so an eval
starting during another job's flush, or after a writer died mid-dump,
read a torn file. OmegaConf turned an empty read into an empty cache;
the libyaml loader on this branch raises StaleCacheError instead.
Writing to a sibling temp file and renaming it over the manifest
leaves readers the old file or the new one. The lock files live under
.locks/, so the rename does not disturb locking, and cache dirs are
setgid with default ACLs, so the replacement keeps the group and
permissions.

Found in the internal review of this branch.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvkM1gKifWmoDPq8Drd4JF

…ler's snapshot

The metrics digest is computed from the snapshot the caller read before
scoring, and the pred context adds positions that appear later to its
own copy. The old test only checked identity (is not); it now adds a
position on demand and checks the caller's dict is unchanged. Dropping
the dict() copy in init_cache_context makes it fail.

Found in the internal review of this branch.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvkM1gKifWmoDPq8Drd4JF

…fy branch

#515 widened the cubic check in metrics_provenance_matches to an
equivalence class; this branch had already moved the prediction stamp
into the same function. Kept both: the class check, then the digest
check. The only textual conflict was the Returns docstring.

#515's new test_equivalent_environment_reuses_the_declared_pins_cache
called the gate without prediction_sources, which this branch makes
required; it now passes _SOURCES, matching the _DIGEST that
_stamp_cubic writes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvkM1gKifWmoDPq8Drd4JF